Multivariate Index Relationship Exploration Using Advanced Machine Learning Techniques

Abstract

Stock market forecasts have always attracted the attention of many analysts and researchers. Popular theory suggests that stock markets are inherently straightforward, and trying to predict them is a trivial matter. It is a difficult problem in itself. The market behaves like a voting machine in the short term, but in the long term it behaves like a pair of scales. Therefore, there is room to predict market movements over a longer period of time. Applying machine learning techniques and other algorithms to stock price analysis and forecasting is a promising area. This white paper begins with a brief overview of the stock market and a taxonomy of stock market forecasting methods. We then highlight some research findings on resource analysis and forecasting. We discuss technical, fundamental, short- term and long-term approaches to equity analysis. Finally, we present some challenges and research opportunities in this area.

Introduction

I. INTRODUCTION

Stock market forecasting and analysis is an attempt to determine the future value of an exchange-traded company's stock or other financial instrument. The stock market is an important part of the country's economy and plays an important role in the growth of the country's industry and trade which ultimately affects the country's economy. Both investors and industry are involved in the stock market and want to know whether stocks will rise or fall over a period of time. The stock market is the primary source for companies to raise funds to expand their operations. It is based on the concept of supply and demand. If demand for a company's shares is high, the company's share price will rise, and if demand for the company's shares is low, the company's share price will fall.

National Stock Exchange of India Limited (NSE) is India's leading stock exchange based in Mumbai. NSE was founded in 1992 as the first demutualized electronic exchange in the country. NSE is the first stock exchange in the country to offer a fully automated, modern, screen-based electronic trading system that offers easy trading opportunities to investors across the country.

NIFTY 50 Index is the National Stock Exchange of India's broad benchmark equity market index for the Indian equity market. It represents a weighted average of 50 Indian corporate stocks across 12 sectors and is one of two major stock indices used in India, the other being the BSE Sensex contains very large datasets that are difficult to extract information from and manually analyze work development as they involve many industries and companies. The application developed in this project not only helps predict the future movement of stocks in the market, but also automates data search, trend analysis, predictive analysis and stock insight generation at the click of a button. Stock market analysis and forecasting reveals market patterns and predicts when to buy stocks. Correctly predicting the future price of stocks can yield big profits.

II. LITERATURE SURVEY

1. Research on Stock Price Prediction Method Based on Convolutional Neural Networks, IEEE 2019 - Sayavong Lounnapha et al. The dataset is taught and tested in relation to the behavior of convolutional neural networks and the Thai stock market. The results show that models based on convolutional neural networks can effectively detect and imagine stock price trends. It provides an important indicator for stock price prediction. The accuracy of the forecast is high, which could also facilitate the financial sector.
2. Enhancing Profit by Predicting Stock Prices using Deep Neural Networks, IEEE 2019-Soheila Abrishami, et al., Economic time series forecasting is a very difficult task and has attracted the attention of many scientists. , is very important for investors. This white paper focuses on introducing a deep learning system that predicts stock values using a set of facts about a subset of stocks on the NASDAQ stock exchange. This model has been trained on the minimum data for a particular stock and in a few steps accurately estimates the final value of that stock. It consists of an autoencoder that removes noise and uses time-series data techniques to syndicate advanced features with original features. These new functions are fed into a stacked LSTM autoencoder to make multi-level predictions of final stock values. Additionally, this estimate is used by the profit maximization approach to provide guidance on the right time to buy or sell a particular stock. The results show that the proposed framework outperforms state-of-the-art time series forecasting methods in terms of analytical accuracy and effectiveness.
3. LSTM Method for Bitcoin Price Prediction: A Case Study Yahoo Finance Stock Market, IEEE 2019- Ferdiansyah et al., Bitcoin is a type of cryptocurrency and currently a type of stock market asset. There are some risks in the stock market
   And Bitcoin is a kind of virtual currency, and in recent years it has gone up and down violently, and it may plummet without knowing the impact on the stock market. Due to its volatility, automated tools are needed to predict Bitcoin on the stock market. This research study explores how LSTMs can be used to make modal predictive Bitcoin stock market predictions. Before confirming the results, the paper tries to measure the results using RMSE (Root Mean Square Error). RMSE is always greater than or equal to MAE. The RMSE metric evaluates how well the model computes continuous values. The method applied in this study to forecast Bitcoin on the Yahoo Finance stock market can predict results above US$12600 for days after the forecast.
4. Stock Prediction Using Machine Learning Techniques, IEEE 2019-Jeevan B et al., Everyone is talking about the stock market these days, and science and business people are interested. This paper mainly uses his RNN (Recurrent Neural Network) and his LSTM (Long Short Term Memory) on the National Stock Exchange, using many factors such as current market prices and anonymous events. It deals with an approach to predicting stock prices using This white paper also describes a recommendation system along with a model based on the RNN and LSTM methods used to select companies.
5. Stock Market Forecasting Using Machine Learning Techniques, IEEE 2020 – Naadun Sirimevan et al. Stock market prices play an important role in today's economy. Researchers have found that social media platforms such as Twitter and internet news tend to influence each individual's decision-making process. This study takes into account behavioral reflexes to web messages in order to reduce the gap and make predictions more accurate. Here, accurate predictions were made for 1 day, 1 week, and 2 weeks later.
6. Machine Learning Stock Market Forecasting, IEEE 2018 – Ishita Parmar, Ridam Arora, Lokesh Chouhan, Navanshu Agarwal, Shikhin Gupta, Sheirsh Saxena, Himanshu Dhiman. In this article, we examine a method [3] for predicting stock prices using regression and LSTM-based machine learning. The factors that are measured are opening price, closing price, low price, high price, and volume. This paper was an attempt to use machine learning techniques to improve accuracy and reliability in determining the future price of a company's stock. The LSTM algorithm produced positive results with higher accuracy in stock price prediction.
7. Stock Prediction Using Machine Learning Techniques, IEEE 2018 – Jeevan B, Naresh E, Vijaya kumar BP, Prashanth Kambli. This paper mainly used [5] long-short-term memory (LSTM) and recurrent neural network (RNN) to predict stock prices, using various factors such as current market prices, price-earnings ratios, and baselines. based on an approach to estimating stock prices for NSE data. and other anonymous events. Model efficiency is analyzed by comparing true and predicted data using RNN plots. Machine learning for predicting stock prices as the model can predict stock prices very close to the actual price. This model captures detailed features and uses different strategies to make predictions. The model trains all NSE data from the web, recognizes and groups inputs, and provides inputs according to user configuration. This RNN-based architecture is highly efficient in stock price forecasting by changing the configuration accordingly and using the backpropagation mechanism to avoid data mixing during data collection and grouping. It has been proven.
8. Stock Market Forecasting Using Machine Learning Techniques, IEEE 2016 - Mehak Usmani, Syed Hasan Adil, Kamran Raza, Syed Saad Azhar Ali. The main objective of this study is to predict the market performance of the Karachi Stock Exchange (KSE) at the end of the day using [6] machine learning algorithms. Predictive models are used to predict various attributes as inputs and predictive markets as positive and negative. Features used in the model include oil prices, gold and silver prices, interest rates, foreign exchange rates (FEX), news, and social media feeds. Machine learning algorithms including Single Layer Perceptron (SLP), Multilayer Perceptron (MLP), Radial Basis Function (RBF), and Support Vector Machine (SVM) are compared. The MLP algorithm, a multi-layer perceptron, performed best compared to other methods. The main feature that helped predict the market was the oil price attribute. The final results of this study confirm that machine learning techniques can predict stock market trends. A multi-layer perceptron machine learning algorithm predicted 70% accurate market performance.
9. Development of Predictive Models for Stock Analysis, IEEE 2017 – R. Yamini Nivetha, Dr. C. Dhaya. A comparative study of three algorithms, namely multiple linear regression (MLR), support vector machine (SVM), and artificial neural network (ANN), is the main purpose of this work. Forecasts are determined by monthly and daily forecasts to forecast the market for the next day. Predict stock prices with sentiment analysis with the best prediction algorithms. A less sophisticated algorithm is the multiple linear regression algorithm that calculates the correlation between volume and price. After research, I found that deep learning algorithms are more developed than his MLR and SVM algorithms.
10. Stock Prediction Based on Information Entropy and Artificial Neural Networks, IEEE 2019 - Zang Yeze, Wang Yiying
    One of the most important elements of the financial system is the stock market. Funds are provided by affiliated investors to support activities and development. Together with information theory and artificial neural networks (ANNs) form a combination of machine learning frameworks. Informational entropy for nonlinear causality and inventory association is also creatively used in this method to facilitate ANN time series modeling. The feasibility of this machine learning framework is analyzed at Amazon, Apple, Google, and Facebook prices. In this paper, we review time series analysis techniques based on information theory and LSTMs for modeling stock price dynamics. Transfer entropy between related variables supporting LSTM time series forecasting is integrated into this modeling infrastructure, so the accuracy of hypothetical results is nearly guaranteed. Although the modeled and actual stock prices are highly correlated, they differ slightly in terms of mean absolute error (MAE) and mean squared error (RMSE), which are examined using the results.

III. METHODOLOGY

The first step in this process is collecting financial data and identifying crashes. We've been looking for daily price information from major stock markets with low correlation. Low cross-correlation is important for effective cross- validation and model testing. To avoid overfitting, we avoided including two datasets with a cross-correlation greater than 0.5 in the collection. Overfitting occurs when a machine learning model tries to cover all data points or exceeds the required data points for a given dataset. Because of this, the model starts caching noise and inaccurate values in the dataset, all of which reduce the efficiency and accuracy of the model. To identify clashes in each dataset, we first calculate the markdown. A drawdown is a continuous drop in price over several consecutive days from the last high to the next low. Then use Emily Jacobson's methodology. In this methodology, a crash in any market he defines as a drawdown at the 99.5% quantile. A quantile defines a specific portion of a data set. H. The quantile determines whether the values in the distribution are above or below certain limits.

Conclusion

We found that returns are based on duration and drawdown and are not dependent on the index\'s daily returns. To identify the decline in each dataset, we first calculated the price decline. A drawdown is a continuous drop in price for several days in a row from the last high to the next low. Simple price patterns defined by long-term price movements and volatility changes seem to occur regularly before crashes. The best models were able to learn these patterns and predict crashes much better than comparable random models. For example, the best regression model for 3-month crash prediction achieved an accuracy of 0.15 and a recall of 0.59 on the test set, whereas an equivalent random model with no predictive power achieved an accuracy of 0.04 and a recall of 0.16. Achieved. The results for 1-month and 6-month crash predictions are similar, with the highest F-beta scores for 6-month prediction and the worst for 1-month prediction. Whether these results are sufficient to optimize investment strategies is debatable. Looking at the test data during the crashes and the price index chart of the crash predictor indicator, some crashes were detected very well, while others occurred with little or no warning from the crash predictor.

References

[1] Chiu, DY, & Chen, P.J. (2009). Dynamically explore the internal mechanisms of the stock market through fuzzy-based support vector machines with high-dimensional input spaces and genetic algorithms. Expert Systems with Applications, 36(2), 1240-1248. [2] Chong, E., Han, C., & Park, F.C. (2017). Deep Learning Networks for Analysis of Stock Market and Forecasting: Methodology, Data Plots, and Case Studies. Expert Systems with Applications, 83, 187-205. [3] Dai, W., Wu, J.Y., and Lu, C.J. (2012). A combination of non-linear independent component analysis and neural networks for predicting Asian stock market indices. Expert Systems with Applications, 39(4),-4452. [4] Das, S.P., & Padhy, S. (2012). Supporting vector machines for predicting Indian stock market futures prices. International Journal of Computer Applications, 41(3). [5] Dash R. & Dash P.K. (2016). A hybrid stock trading framework that integrates technical analysis and machine learning techniques. Journal of Finance and Data Science, 2(1), 42-57. [6] Enke, D. & Thawornwong, S. (2005). Predict stock market returns using data mining and neural networks. Expert Systems with Applications, 29(4), 927-940. [7] Goodhue, D.L., & Thompson, R. L (1995). Adaptability of task technology and individual performance MIS Quarterly, 213-236. [8] Guresen, E., Kayakutlu, G., & Daim, TU (2011). Use of artificial neural network models in prediction of the stock market. Expert Systems with Applications, 38(8), 10389-10397. [9] Kim, K.J. & Lee, W.B. (2004). Stock market forecasting using artificial neural networks with optimal feature transformations. Neural Computing and Applications, 13(3), 255-260. [10] Jasic, T. & Wood, D. (2004). Profitability of Daily Stock Market Indices Based on Neural Network Predictions: A Case Study of the S&P 500, DAX, TOPIX, and FTSE from 1965 to 1999. Applied Finance, 14(4), 285-297. [11] Kim, H.J. & Shin, K. S (2007). A hybrid approach on study of neural networks and genetic algorithms to identify temporal patterns in stock markets. Applied Soft Computing, 7(2), 569-576. [12] Kim, K. J. & Han, I. (2000). A Genetic Algorithm Approaches Feature Discretization in Artificial Neural Networks for Stock Index Prediction Expert Systems with Applications, 19(2), 125-132.

Copyright

Copyright © 2023 Jay Patel, Siddhanth Jain, U. M. Prakash. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.